AD-A117  996  NAVAL  UNDERWATER  SYSTEMS  CENTER  NEW  LONOON  CT  NEW  LO— ETC  F/G  12/1 

EQUIVALENT  BANDWIDTH  of  a  general  CLASS  OF  POLYNOMIAL  SMOOTHERS— ETC! U) 
•.  JUL  82  R  A  LATOURET-TE 

UNCLASSIFIED  NUSC-TR-6601  NL 


NUSC  Technical  Report  6601 


19  July  1982 

Equivalent  Bandwidth  of  a 
General  Class  of  Polynomial 
Smoothers:  With  Application  to 
Bearing  Tracker  Random  Error 
zq  Evaluation 

,,,)  j 

Robert  A.  LaTourette 
Lawrence  C.  Ng 
Submarine  Sonar  Department 

rH 

Q 

>- 

o_ 

s 


Naval  Underwater  Systems  Center 

Newport,  Rhode  Island  /  New  London,  Connecticut 


Approved  for  public  release;  distribution  unlimited. 


C:  n  09  o  0  7 


Preface 


This  report  was  prepared  under  NUSC  Project  No.  A18002.  The  Principal 
Investigator  is  Kenneth  Sargent  (Code  3291).  The  sponsoring  activity  is 
NAVSEA,  Program  Manager  Dr.  Robert  Snuggs  (NAVSEA  PMS  409C). 

The  Technical  Reviewer  for  this  report  was  Dr.  G.  Clifford  Carter,  Code  3331. 


REVIEWED  AND  APPROVED:  19  July  1982 


J.  W.  Kyi* 

Head  Submarine  Sonar 
Systems  Department 

v  'r 


The  authors  of  this  report  are  located  at  the  New  London 
Laboratory,  Naval  Underwater  Systems  Center, 

New  London,  Connecticut  06320. 


REPORT  DOCUMENTATION  PAGE 


RE.*D  I.NSTHICTICWS 
BEFORE  COMPLETI.NG  FORM 


1.  ffCFQNT  NUMMft 

TR  6601 


t.  TITIJ  Mnlln 

EQUIVALENT  BANDWIDTH  OF  A  GENERAL  CLASS  OF 
POLYNOMIAL  SMOOTHERS:  WITH  APPLICATION  TO 
BEARING  TRACKER  RANDOM  ERROR  EVALUATION 


x  namvtrt  catalog  mmt* 


x  type  o#  wort  i  Ptmoo  covered 


«.  nmonmNQ  pro.  report  n umrer 


7.  AUTHOR^ 

Robert  A.  LaTourette 
Lawrence  C.  Ng 


t.  CONTRACT  Off  GRANT  NUMERRm 


9.  PERFORMING  ORGANIZATION  NAME  ANO  AOORESS 

Naval  Underwater  Systems  Center 

New  London  Laboratory 

New  London,  Connecticut  06320 


11.  CONTROUJNG  OPTIC*  NAME  ANO  AOOffCSS 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMREffS 

A18002 


12.  RVORT  GATE 

19  July  1982 


14.  *ONtrOm*G  AGENCY  NAMff  A  AOOffESE  Hf  difftmm  from  ComtrmMm?  Off*** 


IS.  SSCURffY  CLARE.  tmf  thim  nrpmett 

UNCLASSIFIED 


1W.  OffCLAWPfCATIOM  t  DOWNGRADING 
SCHEDULE 


is.  DISTRIBUTION  STATEMENT  tmf  tku  Rmpm*tt 

Approved  for  public  release;  distribution  unlimited. 


17.  OttTfMffUTION  STATEMENT  mf  tkm 


I  u •  Btmr A  JR  if  dkffmrmmt  frmm  Rmpmrn 


1>  ABSTRACT  tCrnmnmmm  mm  iwm  udm  if  MFHMfi'  mmd  idmmnfy  Ay  Afar* 

Random  bearing  error  is  a  major  performance  measure  of  a  sonar  bearing 
tracker.  Programs  currently  employed  in  calculating  random  bearing  error 
from  measured  tracker  bearing  error  data  use  a  standard  polynomial  Least 
Mean  Square  Fit  (LMSF)  algorithm  to  remove  an  unknown  time  varying  mean. 
Previously,  the  effect  of  the  LMSF  algorithm  on  the  residuals  of  the  measured 
tracker  bearing  error  data  was  not  fully  accounted  for.  In  addition,  when 
processing  correlated  bearing  error  residuals,  the  optimum  choice  of  the 
order  of  the  LMSF  and  appropriate  bias  correction  factor  as  a  function.  ■, _ 
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of  signal-to-noise  ratio  (SNR)  were  not  known. 

This  study  investigates  the  properties  of  the  LMSF  in  detail  and  shows 
that  the  LMSF  behaves  as  a  low-pass  filter,  the  frequency  response  character¬ 
istics  of  which  can  be  calculated  exactly.  The  equivalent  noise  bandwidth  of 
the  LMSF  is  shown  to  be  a  function  of  the  sample  size,  the  sampling  time  and 
the  order  of  the  fit.  The  appropriate  bias  correction  factor,  when  processing 
correlated  data,  is  shown  to  be  determined  by  the  ratio  of  the  LMSF  bandwidth 
to  the  equivalent  tracker  bandwidth.  Results  of  the  analysis  are  verified  by 
extensive  simulation.  Finally,  an  operational  procedure  is  given  to  obtain 
an  unbiased  estimate  of  the  variance  for  at-sea  measured  tracker  data. 
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1.0  INTRODUCTION 


Random  bearing  error  is  defined  as  the  standard  deviation  of  bearing 
error  fluctuation  about  the  mean  bearing  error.  Random  bearing  error  is 
obtained  from  recorded  at-sea  test  data  by  analyzing  bearing  error  data 
recorded  over  time  at  a  1-second  rate  while  keeping  relevant  parameters,  such 
as  signal-to-noi se  ratio  (SNR)  and  actual  relative  bearing,  stationary.  In 
practice,  the  above  parameters  are  not  constant.  In  particular,  the  actual 
relative  bearing  and  thus  the  mean  bearing  error  vary  as  a  function  of  time. 
The  time-varying  mean  must  be  removed  prior  to  the  variance  calculation. 

Since  no  statistical  knowledge  of  the  time  varying  mean  is  assumed,  a  standard 
trend  removal  procedure  of  using  a  polynomial  Least  Mean  Square  Fit  (LMSF)  is 
used.  The  LMSF  procedure  causes  bias  in  the  resulting  variance  estimate; 
thus,  in  order  to  obtain  an  unbiased  estimate,  the  classical  LMSF  correction 
factor  is  applied  to  the  sum  of  the  squared  residuals.  However,  this  correc¬ 
tion  is  not  adequate  when  the  measured  tracker  bearing  error  data  is  signifi¬ 
cantly  correlated.  This  is  the  case  when  tracking  in  a  low  SNR  environment. 

This  report  presents  a  detailed  study  on  the  LMSF  procedure,  and  provides 
an  appropriate  bias  correction  for  the  variance  estimate  based  on  the  input 
data  bandwidth  when  processing  correlated  data. 


1.1  DESCRIPTION  OF  PREVIOUS  RANDOM  ERROR  CALCULATION  TECHNIQUE 

The  previous  method  for  calculating  random  bearing  error  assumed  the 
following  input  bearing  error  model: 

Bearing  Error  =  m(t)  +  n(t)  (1) 
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where  n(t)  is  the  desired  tracker  random  bearing  error  output  with  the  follow¬ 
ing  assumed  properties; 

1.  White  sequence  (over  the  sampling  time  of  1  second) 

2.  Zero  mean 

and  m(t)  is  an  unknown  time  varying  mean  with  a  time  constant  much  longer  than 
1  second. 

The  procedure  for  calculating  random  bearing  error  consisted  of  the 
following: 

1.  Select  time  intervals  between  60  and  400  seconds,  where  the  indicated 
SNR  remains  relatively  constant  and  tracker  bearing  error  outputs  indicate 
normal  automatic  tracking. 

2.  Perform  a  6th  order  polynomial  LMSF  on  the  selected  data.  (For  low 
SNR  use  a  Oth  order  polynomial  fit.) 

3.  Calculate  random  bearing  error  from  the  sum  of  the  squared  residuals 
(£*)  of  the  LMSF  by  using  the  following  formula: 

2  £Z  rn\ 

bearing  3  N  -  K  -  1  ^ 

where  N  3  the  number  of  samples 
K  3  the  order  of  the  fit 
E*  3  the  sum  of  the  squared  residuals. 

1.2  PROBLEMS  WITH  THE  PREVIOUS  RANDOM  ERROR  CALCULATION  TECHNIQUE 

The  major  assumption  of  the  previous  random  error  calculation  technique 
is  that  the  polynomial  LMSF  removes  the  unwanted  time  varying  mean  m(t)  while 
retaining  the  desired  unbiased  random  tracker  bearing  error  output  n(t).  The 
key  assumption  is  that  n ( t )  is  effectively  a  white  sequence  for  the  given 
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sampling  time  of  1  second.  In  reality,  the  above  assumption  is  seldom  true. 
However,  at  relatively  high  SNRs  where  the  tracker  averaging  time  is  small, 
the  incurred  bias  is  relatively  small.  At  very  low  SNRs,  where  the  tracker 
averaging  time  becomes  much  longer,  the  biases  will  become  substantial.  The 
technique  of  reducing  the  order  of  the  fit  to  zero  was  necessary  to  reduce 
this  bias  to  an  acceptable  level. 


1.3  TECHNICAL  OBJECTIVE 

The  LMSF  procedure  can  be  viewed  as  a  low-pass  filter.  If  the  equivalent 
cut-off  frequency  of  the  LMSF  procedure  could  be  determined  as  a  function  of 
its  parameters,  then  intelligent  decisions  could  be  made  on  selecting  the 
input  parameters  and  determining  the  appropriate  bias  correction  factors.  In 
the  following  sections,  a  model  relating  the  LMSF  procedure  to  a  low-pass 
filter  will  be  developed  and  analyzed. 
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2.0  THEORETICAL  ANALYSIS  OF  LMSF  PROCEDURE 


In  this  section,  a  description  of  the  LMSF  procedure  as  a  low-pass  filter 
shall  be  developed  under  the  assumption  of  a  zero-mean,  white  input  sequence. 
The  derived  model  of  the  LMSF  as  a  low-pass  filter  shall  then  be  applied  to 
the  case  of  a  zero-mean  correlated  input  sequence.  For  the  correlated  case, 
an  excellent  approximation  shall  be  derived  by  correcting  the  resulting  "sum 
of  squared  residuals"  of  the  LMSF  fit  to  an  unbiased  estimator  of  the  act  1 
input  variance,  a The  above  approximation  assumes  that  the  equivalent  nd- 
width  of  the  LMSF  is  much  smaller  than  the  equivalent  bandwidth  of  tne  c 
lated  input  sequence.  This  relationship  is  shown  as 


BW 


LMSF 


<< 


8W Input  -  At 


where  At  =  sampling  time  in  seconds. 


2.1  THEORETICAL  MODELS 

The  assumed  model  for  the  bearing  error  shall  initially  be  the  same  as 
Equation  (1). 

Bearing  Error  =  m(t)  +  n(t) 

where  n(t)  is  the  desired  tracker  random  bearing  error  output  with  the  follow¬ 
ing  assumptions: 

1.  White  sequence  (over  the  sampling  time  of  At  seconds) 

2.  Zero  mean 

3.  Variance  =  a ^ 

and  m(t)  is  a  deterministic  but  unknown  time-varying  mean  with  a  very  long 
time  constant  compared  to  the  sampling  time. 
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The  LMSF  can  be  modeled  as  a  linear  process  having  the  following  properties: 
Let  RLMSF  be  defined  as  the  "sum  of  squared  residuals"  of  the  LMSF. 


Then 

RLMSF(m(t)  +  n(t) )  =  RLMSF (m(t) )  +  RLMSF (n(t) )  (4) 

provided  that  the  zero  mean  noise  process  n(t)  is  uncorrelated  with  the  time 
varying  mean  m(t) . 

The  only  restrictions  concerning  the  form  of  the  fitting  function  are 
that  it  be  selected  from  a  set  of  complete  functionals  and  that  a  constant 
term  be  included. 

Since  the  LMSF  is  a  linear  process,  its  effect  on  m(t)  +  n(t)  can  be  ana¬ 
lyzed  separately.  Considering  the  effect  of  the  LMSF  on  the  unwanted  time- 
varying  mean  separately  would  dictate  that  a  higher  order  fit  be  used  in  order 
to  remove  the  unwanted  mean  completely.  Unfortunately,  this  also  removes  a 
portion  of  the  noise  process.  With  this  in  mind,  we  shall  now  look  at  the 
effect  of  the  LMSF  on  the  input  noise  process,  n(t).  In  a  later  section  we 
shall  return  to  analyze  the  choice  and  effect  of  the  LMSF  procedure  on  the 
unwanted  mean,  m(t). 


2.2  EQUIVALENT  LMSF  BANDWIDTH  CORRECTION  -  FREQUENCY  DOMAIN  APPROACH 

Let  [H(f ) | 2  be  the  power  transfer  function  of  a  digital  low-pass  filter, 
operating  at  a  sampling  interval  of  At  seconds.  Let  o^  be  the  variance  (or 
power)  of  a  zero-mean  white  sequence.  The  expected  power  spectral  density 
(SRD)  of  the  above  process,  sampled  at  intervals  of  At  seconds  is  given  by: 

SF>b  =  At  (5) 

which  is  uniform  over  the  frequency  band. 


6 


TR  6601 

The  expected  power  at  the  output  of  the  low-pass  filter,  given  the  above 
zero-mean  white  sequence,  is  given  by: 


l 

f  &  U/ft  2  On 

Wn-I  W'-T 

J  n  a+ 


°?ii  =  aA  ‘  At  *  IW 


Jyp-  *  / 


|  H ( f ) | 2df 


However,  the  equivalent  two-sided  bandwidth  (BWg)  can  be  written  as 


BWe  = 


I H ( f ) | 2df 


Note  that  we  have  implicitly  assumed  in  Equation  (7)  that  the  input  sequence 
is  sampled  at  or  above  the  Nyquist  rate. 

Therefore,  combining  Equation  (6)  and  Equation  (7)  yields 

ofil  *  •  At  •  BWg  (8) 

Let  I7  be  the  expected  value  of  the  "sum  of  squared  residuals"  of  a  kth  order 
LMSF  to  a  white  sequence  of  variance,  a^.  The  input  sequence  is  sampled  N 
times  at  intervals  of  At  seconds.  The  LMSF  function  contains  K  +  1  terms. 
Also,  let  the  expected  standard  variance  aj  be  defined  as: 


JS  "  N 
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However,  regression  theory  [1]  states  that 


r2 

N  -  k  -  1 


is  an  unbiased  estimator  of  o^; 


therefore 


(10) 


The  expected  variance  1  of  the  LMSF  filter  with  respect  to  the  white  input 
sequence  can  be  written  as 


^il=aA-^f  (11) 

Figure  1  illustrates  the  above  equation. 


Using  Equation  (9)  and  Equation  (10)  in  Equation  (11)  yields 


°fi1 


or 


K  +  1 
R 


(12) 


Equating  Equation  (8)  with  Equation  (12)  yields  an  equation  for  the  (two- 
sided)  equivalent  bandwidth  for  the  LMSF  low-pass  filter,  specifically; 

BWe(lHSF)  ■  {j-fij-  (13) 

Note  that  the  only  assumptions  made  relating  to  the  form  of  the  fitting 
functions  were  that  they  were  selected  from  a  functionally  complete  set  and 
that  they  include  a  constant  term.  This  implies  that  the  effect  on  the  vari¬ 
ance  estimate  from  the  input  white  sequence,  within  the  above  restriction,  is 
on  the  average  exactly  predictable  and  independent  of  the  choice  of  fitting 
functions.  However,  the  choice  of  fitting  functions  can  have  a  drastic  effect 
on  the  removal  of  the  unwanted  mean  m(t).  The  above  property  can  be  used  to 
advantage,  as  shall  be  discussed  later. 
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Let  us  now  assume  we  have  a  correlated,  zero-mean  input  sequence.  We 
shall  model  the  above  sequence  as  the  output  of  a  discrete  low-pass  filter, 
with  sampling  time  At  seconds  and  driven  by  a  unit  variance  zero-mean  white 
input  sequence.  If  the  power  spectral  density  function  of  the  low-pass  filter 
is  given  by  J W ( f ) } 2 ,  then  the  variance  of  the  correlated  sequence,  a is 


i  W (f ) | 2df 


(14) 


The  (two-sided)  equivalent  bandwidth  of  the  input  correlated  sequence  is 
defined  by 


1 

8We( Input)  =  |W(f )  |2df  (15) 

'  0 


It  is  assumed  that 


BWe(LM$F)  «  BWg( Input)  <  -^  (16) 


The  objective  is  to  find  a  correction  factor,  C,  relating  the  expected  value 
of  the  standard  variance  (o^  =  E^/N)  of  the  LMSF  to  the  actual  variance  a ^  of 
the  correlated  zero-mean  input  sequence 


°A  =  C  *  °S 


(17) 


The  expected  measured  variance,  o|,  at  the  output  of  a  low-pass  filter  H(f) 
for  the  given  correlated  zero-mean  input  sequence  can  be  written  as 


|W(f)|2df 


(18) 
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or,  using  Equation  (14): 


Substituting  Equation  (15)  into  Equation  (19)  yields 


(19) 


TWI 


i 


J. 

At 


Ml 

(f) 1 

2 

Ml 

!<5)| 

T 

|H(f)|2df 


BWg( Input) 


(20) 


Assuming  that:  1)  |W(f)|2  is  reasonably  flat  at  low  frequencies,  2)  roll-off 
of  |H(f )  | 2  is  rapid,  and  3)  the  equivalent  bandwidth  of  [ H ( f ) | 2  is  much 
smaller  than  the  equivalent  bandwidth  of  |W(f)|2,  the  above  integral  can  be 
approximated  by  the  following: 


|H(f ) | 2df 


I H ( f ) | 2df  *  BWe(fil) 


(21) 


Note  that  if  | H ( f ) | 2  =  6(f),  an  impulse  response  in  frequency  (which  is  the 
case  for  a  zero-order  LMSF),  then  the  above  approximation  is  exact. 

Substituting  this  result  into  Equation  (20)  yields 


BWe(fil) 

‘  ( Input ) 
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or 

•  of  (22) 

Substituting  the  equivalent  bandwidth  of  the  LMSF  filter  BWg(LMSF),  Equation 
(13)  for  the  equivalent  bandwidth  of  the  low  pass  filter,  BWg(fil),  yields 

(23) 

Finally,  comparing  Equation  (23)  and  Equation  (17)  we  have  the  desired  approx¬ 
imation  to  the  correction  factor,  C. 


Computer  analysis  has  indicated  that  the  polynomial  LMSF  filter  does  have  a 
reasonable  roll-off  of  approximately  6  dB/octave.  Therefore,  if  the  low-pass 
filter  model  for  the  input  correlated  noise  sequence  is  reasonable,  then  the 
above  approximation  is  very  good  for  BWg(LMSF)  «  BWg(Input)  <  1/At.  As 
noted  earlier,  for  the  case  where  K  *  0  the  approximation  is  exact. 

2.3  EQUIVALENT  LMSF  BANDWIDTH  CORRECTION  —  ANALYTICAL  TIME  DOMAIN  APPROACH 

Guided  by  the  intuitive  approach  as  described  in  Section  2.2,  this 
section  analytically  re-examines  the  problem  of  obtaining  an  unbiased  esti¬ 
mate  of  the  variance  from  finite  observations  (or  measurements)  of  a  non¬ 
stationary,  correlated,  random  process.  For  the  most  part,  the  analyses  shown 
in  this  section  support  the  intuitive  results  as  presented  in  Section  2.2.  The 
developments  in  this  section  show  that:  1)  LMSF  behaves  as  a  low-pass  filter 
whose  bandwidth  is  determined  by  the  triplet  (K,  N,  At)  where  K  is  the  order 
of  the  LMSF,  N  is  the  number  of  samples,  and  At  is  the  sampling  time;  and  2) 
correlated  noise  input  and  the  subsequent  application  of  the  LMSF  to  remove 
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non-stationary  mean  results  in  biases  in  the  variance  estimates.  The  analysis 
shows  that  in  order  to  obtain  an  unbiased  estimate  of  the  variance,  a  correc¬ 
tion  factor  which  accounts  for  the  correlated  data  and  the  effect  of  the  LMSF 
is  needed. 

The  following  analyses  assume  no  statistical  knowledge  of  the  non- 
stationary  mean,  except  that  it  is  a  slowly  varying  function  of  time  and  is 
representable  by  a  finite  order  set  of  functions  (for  example,  polynomials). 

In  order  to  provide  a  proper  perspective  of  the  analysis,  the  unbiased  esti¬ 
mate  of  the  variance  of  a  stationary  process  with  correlated  data  sequence 
will  be  discussed  first.  After  an  unbiased  correction  factor  has  been 
derived,  we  then  consider  the  further  complication  of  non-stationary  mean. 


2.3.1  Unbiased  Variance  Estimate  from  Stationary  Correlated  Data 

Let  Xp  x^,  ...,  xN  be  a  sequence  of  N  observations  (or  measurements) 
from  a  random  process,  the  mean  and  covariance  of  which  are  given  by 


E [x^ ]  *  m  i  =  1,  2,  ...»  N 

E[(xi  -  "0(Xj  -  "01  *  aVjj.  V  j,  j 


(25) 


where  p ^ j  is  the  normalized  covariance  coefficient  between  samples  x^  and  x^. 
What  is  now  sought  is  an  unbiased  estimate  of  the  mean  m  and  the  variance  a*  of 
the  random  process  from  the  observed  sequence  {xn},  n  ■  1,  2,  ...,  N.  If  the 
data  is  uncorrelated;  i.e.,  p. .  =  6. .,  then  it  is  well  known  [2]  that  the 

*  J  *  J 

standard  unbiased  estimates  of  the  mean  and  variance  are  given  by: 

N 

7  *  \  m  xi  (26) 

i=l 


u0aS 


(27) 
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where 

N 

°|-h  E  <*t  -*>2  <28> 

i=l 

and 

"0  ■  T^T  I29) 

1  ■  ft 

are  defined  as  the  standard  estimate  of  the  variance  and  the  required  unbiased 
estimate  correction  factor,  respectively.  It  will  be  shown  later  that  u0  is 
the  unbiased  estimate  correction  factor  for  a  zero-order  LMSF. 

However,  if  the  data  is  correlated,  as  is  the  case  at  present,  the  esti¬ 
mated  mean  7  remains  unbiased  whereas  the  variance  becomes  biased.  Further¬ 
more,  Equation  (27)  always  underestimates  the  true  variance.  This  can  be 
shown  readily  as  follows: 

Taking  the  expectation  of  Equation  (27),  we  obtain 
E[a2]  =  UqE[o|] 

(  N 

=  WoE  \  N  Z  Kxi  -  m)  -  U  -  m)l2 

(  1-1 

!n  n 

n-  Z  (xi  - rn)Z  - 1  Z  (x 

i=l  i=l 
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u0 


J2  -  Jr  L  21  Pij 
1  j 


r  s.p 


ij 


VQ  \1  -  J - fc- 


for  al 1  i  and  j . 


(30) 


The  effective  number  of  independent  samples  are  defined  as 


N  A  N2 

e  £  P 


ij 


i  J 


It  can  now  be  seen  that 


(31) 


Ne  = 


N  if  p. ,  =  6(i  -  j)  V  i,  j,;  i.e.,  uncorrelated 
*  J 


1  if  pH  *  1  »  1,  j,;  i.e.,  highly  correlated 

’  J 


(32) 


Therefore,  Equation  (30)  can  be  written  as: 


E[o2]  =  u0^l  -  ^ 


1  -f 


nr 

-r  J°2 


(33) 


Since  from  Equation  (32)  Ng  <  N,  we  have 


i  -i  i- 


<  1 
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thus,  in  general 


E [o2l  <  a2 


and 


E [a2]  =  a2  (34) 

if,  and  only  if,  the  data  sequence  is  uncorrelated  (or  white). 

Thus,  it  is  shown  that  the  standard  unbiased  estimate  of  variance  from  a  cor¬ 
related  data  sequence  becomes  biased  and  the  true  variance  is  always  under¬ 
estimated.  The  unbiased  estimate  of  the  variance  is  defined  as  follows; 


V0°S 


where 


v0  " 


1 


(35) 


(36) 


is  the  unbiased  correction  factor. 


[f  the  equivalent  bandwidth  of  the  correlated  noise  is  defined  as  B^, 
then  for  T  second  observation  time,  we  have 


N 


e 


TB 


N 


=  N  •  At  •  Bn 


(37) 
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and  so  the  unbiased  correction  factor  becomes: 

1 


v0 


1 


T 


•  At  •  B 


N 


1  - 


(38) 


where  is  the  equivalent  bandwidth  of  the  zero-order  LMSF,  as  will  be 

shown  later. 

This  is  identical  to  Equation  (24)  where  a  zero-order  fit  is  assumed.  Thus 
Equation  (38)  indicates  that  if  one  can  estimate  the  equivalent  bandwidth  of 
the  random  process  and  calculate  the  equivalent  bandwidth  of  the  LMSF,  then  an 
approximate  unbiased  correction  factor  on  the  standard  variance  calculation 
can  be  found.  Note  that  in  the  following  sections  the  unbiased  correction 
factors  are  denoted  by  either  ^  or  where  uk  is  the  correction  factor  for 
the  kth  order  fit  with  white  noise,  and  is  the  correction  factor  for  the 
kth  order  fit  with  correlated  noise. 


2.3.2  Unbiased  Variance  Estimate  from  Non-Stationary  White  Data 

It  is  assumed  the  non-stationary  component  of  the  random  process  is  due 
to  the  time  varying  mean.  It  is  further  assumed  that  the  mean  variation  can 
be  modeled  by  a  finite  set  of  functions  in  time.  The  LMSF  technique  is  used  to 
remove  the  non-stationary  component.  The  noise  process  is  assumed  white,  and 
an  unbiased  estimate  of  the  variance  from  a  finite  sequence  of  observations  is 
sought. 

It  will  be  shown  that  LMSF  behaves  as  a  low-pass  filter,  the  filter 
frequency  response  of  which  can  be  controlled  by  the  triplet  (K,  N,  At).  An 
expression  will  be  derived  which  indicates  the  resulting  accuracy  of  the  vari¬ 
ance  estimate. 
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Let  fx-},  i  =  1,  2,  ....  N  be  a  sequence  of  N  observations  from  a  random 
process,  the  mean  and  variance  of  which  are  given  by: 

E  [  x i ]  =  m. 

E [ ( x i  -  m-  )(Xj  -  nij)]  =  0*6^  (39) 

where  6..  is  the  kronecka  delta  function.  It  is  assumed  that  mi  can  be 
modeled  by  a  finite  order  polynomial;  namely: 

K 

m(t)  =  aktk 
k=0 

or  for  discrete  time 

K 

m1  =  2  Vi  i  =  1,  2,  ....  N  (40) 

k=0 


Note  that  for  convenience  we  have  modeled  the  time  varying  mean  by  polyno¬ 
mials.  However,  the  results  presented  herein  are  applicable  to  any  other 
polynomials.  For  example,  one  obtains  a  trigonometric  polynomial  based  on  a 
Fourier  series  representation. 


Now  let 


ii  *  ti  t,  t»  ...  tfjT, 


—  =  fa0  ***  aN^ 


(41) 


where  [  ]  denotes  the  transpose  of  a  vector. 


Then,  the  time  varying  mean,  Equation  (40)  can  be  written  in  vector  form  as: 

mi  =  tja  (42) 
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The  N  observations  can  be  written  as: 


Xj  =  m-  +  w- ;  i  =  1,  2,  . . . ,  N 


(43) 


where 


wi 


-  mi 


with 


Efw,]  =  0,  Efw^j)  =  a26(  i  -  j) 
Equation  (43)  can  be  written  in  matrix  form  as: 


or  more  concisely  as: 


wi  th 


— 

“*  — 

""  — 

ltj  ...  tj 

a0 

W1 

1  t2  ...  t2 

51 

• 

• 

• 

• 

• 

• 

It  t^ 

ak 

WN 

—  — 

X  =  M  +  W 


(44) 


(45) 


M  =  Ha 


where  H  is  a  matrix  of  dimension  Nx(K  +  1). 
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1 


The  parameters  of  interest  are  the  unknown  coefficients,  a.  The  method 
of  least  square  [4]  minimizes  the  following  quadratic  expression: 

J  =  (X  -  Ha)T(X  -  Ha) 

with  respect  to  the  unknown  coefficient  vector,  a^  The  resulting  solution  is 
given  by: 

a  =  LX;  where  L  =  (HTH)‘1HT  (46) 

and  L  is  a  known  linear  operator. 

Using  Equation  (45),  Equation  (46)  can  be  written  as: 

a  =  LM  +  LW  (47) 

Thus,  the  LMSF  is  a  linear  process. 


/\  A  A 

Since  a  depends  on  the  noise  vector  W,  a  is  a  random  vector.  However,  a  is  an 
unbiased  estimate  of  a  since  (from  Equation  (47)): 


E [a]  *  LM 

=  (HTH)_1HTHa 


*  a 

Furthermore,  the  covariance  of  a  is  given  by: 

E[(a  -  a)  (a  -  a)T]  =  E[LWWTLT]  =  (HTH)_1a2 


(48) 


(49) 


The  estimated  mean  is  given  by: 


M 


Ha 
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and  the  error  residual  is: 


=  H(a  -  a)  +  W  (50) 

The  standard  estimate  of  the  variance  is  given  by  the  averaged  sum  of  the 
square  of  the  residuals;  i . e . : 


N 


i  =  l 


=  jjj-  Trace(eeT) 

=  1  Trace{H(a  -  a) (a  -  a)THT  +  WW T  +  H(a  -  a)WT  +  W(a  -  a)THT}  (51) 

N 

where  for  a  given  matrix.  A,  Trace(A)  =  ^  a^ . 

i  =1 

In  order  to  evaluate  the  quality  (amount  of  bias)  of  the  estimate  given  by 
Equation  (51),  the  expectation  on  both  sides  of  the  equation  is  taken,  which 
yields: 

£[a|l  =  J  Trace{HE((a  -  a) (a  -  a)T)HT  +  E[WWT)  +  HE [(a  -  a)WT] 

+  ETW(a  -  a)THT]}  (52) 

Recall  from  Equation  (47)  that: 

a  =  (HTH)"1HT[M  +  W] 

=  a  +  (HTH)'‘HTW 
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then  the  following  relations  can  be  derived: 

E[(a  -  a) (a  -  a)T]  =  (HTH)_1a2 

EfWWT]  =  a2IN 

(53) 

E[(a  -  a)WT]  =  -(HTH)_1HTa2 

E[W(a-a)T]  =  -H(HTH)-1a2 

Using  the  relations  of  Equation  (53)  in  Equation  (52)  yields: 

E [a|]  =  jp  Trace{lN  -  H(HTH)_1HT}  (54) 

but 


Trace{lN}  =  N 

Trace{H(HTH)'1HT}  =  Trace{HTH(HTH)-1 } 

=  Trace{IK+1}  =  K  +  1 

where  IK+^  denotes  the  identity  matrix  of  dimension  K  +  1,  and  K  is  the  order 
of  the  LMSF.  Finally,  because  the  Trace  operator  is  a  linear  operator  Equa¬ 
tion  (54)  becomes: 


E 


We  form  the  unbiased  Kth  order  fit  estimates  by  forming: 


'Ll  N 

aubK  ~  N  -  K  -  1 


a 


2 

s 


*  ^ffs 


(55) 
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where 


“K°.  Tn  (56) 

1  N 

is  the  unbiased  correction  factor  for  the  Kth  order  fit  with  uncorrelated 
data.  It  will  be  shown  in  the  next  section  that  (K  +  l)/(NAt)  is  simply  the 
equivalent  bandwidth  of  the  Kth  order  LMSF.  Note  that  Equation  (56)  is  the 
well-known  result  found  in  the  study  of  the  analysis  of  the  variance  [1]. 
With  K  =  0  (zero-order  LMSF)  Equation  (56)  reduces  to  Equation  (29). 

2.3.3  Equivalent  Bandwidth  LMSF  in  the  Presence  of  White  Data 


It  is  well  understood  that  an  LMSF  behaves  as  a  low-pass  filter.  However, 
not  so  obvious  are  its  frequency  response  characteristics.  This  section 
derives  the  frequency  response  of  the  LMSF.  In  particular,  the  relationship 
of  the  LMSF  with  the  triplet  (K,  N,  At)  will  be  analyzed.  The  equivalent 
bandwidth  (two-sided)  of  the  LMSF  is  derived  for  the  case  of  a  white  noise 
sequence. 

Our  approach  to  calculating  the  LMSF  equivalent  bandwidth  is  as  follows: 

1.  Treat  the  LMSF  as  a  classical  filter,  the  transfer  function  of  which 
is  sought  to  be  identified. 

2.  Observe  the  spectral  response  of  the  output  sequence  when  the  LMSF 
filter  is  driven  by  a  white  input  sequence. 

3.  Take  the  ratio  of  the  output  spectral  response  to  the  input  spectral 
response  yielding  the  magnitude  square  response  of  the  LMSF  filter. 

4.  Integrate  the  magnitude  squared  response  to  yield  the  equivalent 
bandwidth . 
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P(£)  =  E{jM(i)|2} 

(  N  N  2it1 

■El  £  (tjLMW^Je-J  nr 

|  n  m 

»  i 

N  N  2vl 

=  a2£  £  anrne-J  IT  <n  ’  m>  ;  l  -  0,  1,  2 . N  -  1  (59) 

n  m 

where 


■  t>VV 

If  desired,  anm  can  be  factored  as  described  in  the  following  paragraphs. 

Since  (HTH)'1  is  a  square  non-singular  matrix,  it  can  be  factored  into  a 
product  of  two  matrices;  i.e.,  one  can  write 

(HTH)"1  =  DTD 


where  D  is  an  upper  triangular  matrix.  Therefore: 


a  *  tVot_ 

ntn  — n  — m 


where  the  column  vector  8m  3  Dt^. 


(60) 
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Since  the  average  input  spectrum  response  is  E{|X(i,)|2}  =  No2  for  all 
frequency  l,  the  average  magnitude  square  response  of  the  LMSF  filter  is: 

n  m 


M 

-tE 

n=l 


./N 


Bne 


-J 


2rrZn 

N 


2 


or 

G(Z) 

where  G ( 2.)  is  the  equivalent  transfer  function  of  the  LMSF  filter. 

The  equivalent  bandwidth  of  the  LMSF  filter  can  be  determined  by  integrating 
the  filter  power  spectrum  and  dividing  by  the  filter  maximum  power  level. 
Thus, 

N-l 

£  igu)i2 

Bf  ■  '  (62) 

G(0)  is  the  maximum  since 

N 

|B„I  -|G(0)|  U. 


|G(l)|. 


N 

4  v 


Sne 


2niln 

N 


-  4  £  sr 


2irJ,n 


t  «  0,  1,  2, 


N  -  1 


(61) 
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Now  applying  Parseval's  Relation  [3]  yields: 


N-l 

Z  |G(H)i2 

n=o 

N-l 

=  Z  E{  |M(JL)  |2} 

-  h*  Z  Eft) 

n=l 

-  E  | 
°  1 

N 

Y.  m2 

i—t  n 

n=l 

=  JT  e{mtm} 

=  ~t  Trace{E(MMT)} 

=  jr  Trace{E(HaaTHT)} 
=  Trace{H(HTH)',HT} 

=  K  +  1 

also, 

|S(°)i2  *  E{|M(0)|2J 


=  ^  sum[H(HTH)"lHT] 


=  1 


(63) 
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where  for  any  matrix.  A,  sum(A)  =5  ?aii‘  The  Pro°f  that  sum[H(H^H)*lH^]=N 

'  J  J 

in  showing  |G (0 ) | 2  =  1  for  white  noise  input  is  shown  in  Appendix  A. 

Therefore,  the  equivalent  bandwidth  (two-sided)  of  LMSF  with  white  noise 
sequence  is: 


=  (K  +  1)  Af  (Hertz) 

where  Af  =  is  the  OFT  frequency  sampling  interval. 

Thus,  we  obtain  the  desired  result  of  the  equivalent  bandwidth  as  a  func 
tion  of  the  triplet  (K,  N,  At). 

Bf =  tist  (Hertz)  (64) 

This  result  is  intuitively  pleasing  since  for  a  fixed  sample  size,  N,  increas 
ing  K,  the  order  of  the  fit  will  pass  more  noise  corresponding  to  a  larger 
bandwidth. 


2.3.4  Unbiased  Variance  Estimate  from  Non-Stationary  Correlated  Data 

We  assume,  as  before,  that  the  non-stationary  component  of  the  random 
process  is  due  to  the  time-varying  mean,  which  can  be  modeled  by  a  finite- 
order,  algebraic  polynomial.  The  noise  process,  however,  is  assumed  to  be 
correlated,  and  an  unbiased  estimate  of  the  variance  from  a  finite  set  of 
observations  is  sought. 

Let  (X^;  i  =  1,  2,  ...,  N  be  a  sequence  of  N  observations  from  a  random  pro¬ 
cess,  the  mean  and  variance  of  which  are: 

E [X]  =  M 

E[(X  -  M)(X  -  M)T]  =  o2R 
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where  =  [x^,  X2,  x^]T  and  R  is  the  normalized  covariance  matrix.  As 
was  shown  in  the  last  section,  the  data  can  be  written  in  vector  form  as: 

X  =  M  +  W  (65) 


where 


E  [W]  =0 

E[WWT1  =  a2R  (66) 

Assuming  that  the  time-varying  mean  can  be  modeled  by  a  polynomial  or 
appropriate  set  of  functionals,  then 

M  =  Ha 

where  a  and  H  are  defined  as  in  Equation  (45). 

The  method  of  least  square  yields  an  estimate  of  the  polynomial  coeffi¬ 
cients,  given  by: 


a  =»  (HTH)'lHTX 


It  can  be  seen  that: 


E[a]  *  E{(HTH)"1HTa  +  (HTH)“1HTW} 
*  a 


and 


E[(a  -  a) (a  -  a)T]  *  (HTH)‘lHTRH(HTH)*1 


*  LRL 


T 


(67) 
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where 

L  I  (HTH)_1HT 

The  error  residual  is: 

e  =  H(a  -  a)  +  W  (68) 

and  the  standard  estimate  of  the  variance  gives: 


N 


i  =  l 

=  jq-  Trace  (eeT} 

=  J  Trace{H(a  -  a) (a  -  a)V  +  H(a  -  a)WT  +  W(a  -  a)THT  +  WW T }  (69) 

To  investigate  the  bias  of  the  variance  estimate,  the  expected  value  on  both 
sides  of  Equation  (69)  is  taken,  and  yields: 

E[a|]  =  Trace{HlRLTHT  -  HLR  -  RLTHT  +  R}  (70) 

By  noting  the  relations 

Trace{R}  =  N 

Trace{HLR}  =  Trace{RLTHT} 

Trace{HLRLTHT}  =  Trace{HlR} 

Equation  (70)  becomes: 

E[a|l  -  fi  IH  -  Trace (HER H  (71) 
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We  can  form  the  unbiased  Kth  order  fit  estimate  of  the  variance  from  corre¬ 
lated  data  as: 


°ubKC  =  vKffS 


(72) 


where 


Trace  [H(HTH)_1HTRi 
N 


(73) 


.Note  for  uncorrelated  noise,  R  =  I,  and  Trace [H(HTH)~ 1HT]  =  K  +  1;  therefore 
Equation  (73)  reduces  to  Equation  (56)  and: 


=  = 


1  - 


1 

K  +  1 
N 


For  zero-order  fit. 


Trace[H(HTH)‘lHTR]  =  £  L 

i  J 


Equation  (73)  reduces  to  Equation  (36)  with 


For  the  general  case,  let  R  »  I  +  Rg 


where  Rg  is  the  matrix,  R,  with  zeros  replacing  the  diagonal  elements.  The 
following  can  then  be  written  as: 

Trace{H(HTH)'‘HTR}  *  Trace{IK+1}  +  Trace{H(HTH)' lHTRQ} 


*  K  +  1  +  A 
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where 


A  £  Trace {H(HTH)-1HTR0} 


The  correction  factor  can  then  be  written  as: 

K  +  1  +  A 


1  - 


IT 


,  K  +  1 
i - m— 


-  i 


(74) 


where 


N1  £ 
e 


1  + 


(75) 


r~ri 


is  defined  as  the  effective  number  of  independent  samples. 


Mote  that  Ng  is  determined  by  the  order  of  the  fit  and  the  data  correlation 
characteristic.  Also  note  that  in  general  is  different  from  Ng,  the  effec¬ 
tive  number  of  independent  samples  due  for  a  zero-order  LMSF.  On  the  other 
hand,  the  equivalent  bandwidth  of  the  LMSF  with  uncorrelated  data  is  given  by 
Equation  (64)  as: 


B 


f 


_  K  +  1 
'  NAt 


(76) 


and,  prior  to  the  LMSF  procedure,  the  correlated  data  has  a  bandwidth,  Bn,  and 
the  effective  number  of  samples,  N  .  Furthermore,  Bn  and  Ng  are  related  by: 

N0  ■  NAtBn  (77) 


Using  relations  Equations  (76)  and  (77)  in  Equation  (74)  yields: 
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where 


\  = 


1 


K  +  1 
NAt 


At 


1 


l 


e 


1  + 


A 

K  +  1 


) 


accounts  for  the  correction  on  due  to  correlated  data. 


(78) 


The  remainder  of  this  section  investigates  the  behavior  of  the  equivalent 
bandwidth  correction  factor,  e,  as  shown  in  Equation  (78).  For  convenience, 
they  are  restated  as  follows: 

where 


2  2  °u 

i  J 


A  =  Trace[H(HTH)'1HTR]  -  (K  +  1) 

and  R  is  the  correlation  matrix,  the  ith  ,and  jth  elements  of  which  are  given 
by  o . j .  Using  results  developed  in  Appendix  A,  the  following  extreme  cases  of 
correlation  can  be  evaluated. 
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Case  (1)  -  Data  uncorrelated:  p. .  =  6--:  Vi,j 

I  J  I  J 

Therefore  R  =  I,  and  Trace  {H(h"'"h)"1h"'"r}  =  K  +  1;  furthermore 


N  N 


N  N 


23  pij  :  ^  ^  6- j  =  N,  Hence,  e  =  1  is  obtained;  Namely,  no  correction 
i  j  i  j 

is  necessary. 

Case  (2)  -  Data  totally  correlated:  p,.  =  1:  Vi ,  j 

1  J 

For  this  case,  Ng  =  1,  and  R  is  a  matrix  with  unity  elements.  It  can  then  be 
shown  that  (using  relations  obtained  in  Appendix  A): 

Trace{H(HTH)"1HTR}  =  Trace{R}  =  N 


A  »  N  -  (K  +  1), 


e  B  N  1  +  K  + 


1  1  +  N-(KM] 

n  1  k  +  t 


1 

K  +  1 


Therefore,  for  any  sequence  with  <  orrelation  coefficient  p,  the  correction 
factor,  e(p),  must  lie  between  these  two  extreme  cases;  i.e.: 

K-rrie(p)l1  (7? 

where  K  is  the  order  of  the  fit. 
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Thus,  combining  Equations  (78)  ana  (79),  it  is  concluded  that  in  the  presence 
of  correlated  data,  the  effective  bandwidth  of  the  LMSF  decreases  with  respect 
to  the  case  of  uncorrelated  data. 


2.3.5  Equivalent  Transfer  Function  of  LMSF  in  the  Presence  of  Correlated 
Data 

In  this  section,  the  expression  for  the  equivalent  LMSF  transfer  func¬ 
tion  in  the  presence  of  correlated  data  is  derived.  Our  approach  to  calculate 
the  LMSF  equivalent  transfer  function  is  as  follows: 

1.  Assume  the  LMSF  behaves  as  an  ordinary  filter  for  a  given  batch  of 
data  sequences, 

2.  Observe  the  output  spectral  response  for  a  given  input  spectral  dis¬ 
tribution,  and 

3.  Obtain  the  magnitude  square  filter  response  of  the  LMSF  by  taking  the 
ratio  of  the  output  spectrum  over  the  input  spectrum. 

Let  {Xi},  i  =  0,  ...,  N  -  1  be  a  random  input  sequence  with  statistics: 

E  [x- ]  =  0  Vi 

E[xiXj]  =  cr2Pij-  V  i , j  (80) 

where  p.j  is  the  normalized  cross-correlation  coefficient. 

Let  X ( 2.)  denote  the  DFT  of  {x i }  so  that: 


XU) 


N-l 

£  xne'J  TT  " 


n=0 


(81) 
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Then,  the  averaged  power  spectrum  of  the  input  sequence  at  frequency  l  is 
given  by: 


GU)  =  E{|X(Jl)|2} 


=  E{X(£,)X*(£) } 


N-l  N 

=  EE 

n=0  m=0 


■J  ^  ("  -  "0 


=  a‘ 


N-l  N_1  2tt  SL 

E  Z  VJ  t  <"  -  -> 

N=0  m=0 


(82) 


•  2tt£ 


N 


where  R  =  [ p -  - ]  is  the  correlation  matrix,  the  vector  V  =  1,  e" 

2iri  /N  .  1)]  T  *  1 

e  J  tl  v  '  ]  ,  and  is  the  complex  conjugate  transpose  of  V^.  Since 

the  correlation  coefficient  is  symmetric;  i.e.,  Pmn  =  pnm.  Equation  (82) 
can  be  simplified  ^to: 


Ga(£) 


|  N-l 

=  a2  (  N  +  2  2  (N  -  K)p(K)cos  k) 
'  K=1  ' 


/  N-i  j 

=  No2  1  +  2  Yj  U  -  ^)p(K)cos^k)  J 

(  K=1  '  '  ) 


(83) 


where  it  is  assumed  that  the  sequence  is  wide  sense  stationary  so: 
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On  the  other  hand,  the  output  sequence  from  the  IMSF  is  given  by  {y^},  i  =  0, 
2,  N-l,  and,  using  the  vector  notation: 


l  =  Ha 


where  a_  is  the  LMSF  solution  of  the  unknown  coefficient  vector.  Equivalently, 
the  output  sequence  can  be  written  as: 


*i  =  M 

where  h_j  =  (1  t-  ...  t1^]  is  the  ith  row  of  the  matrix  H. 


Using  Equation  (67),  the  following  is  obtained 

T 


E [yi ]  =  hjE[a]  -  0 

T  T  T 

Ety^jl  =  hjE[aa]hj  =  a2^LRL  'hj 


where  L  =  (HTH)_1HT. 


Letting  Y(i)  be  the  OFT  of  the  sequence  {y^, 


then 


(84) 


,  Mn 

YU)  *  £  yne"J  N 
n*0 


and  the  output  spectrum  is  given  by  the  average  magnitude  square  of  YU), 


or 


GyU)  =  E{  | Y(Z)  |  2 } 


36 


TR  6601 


Let  Kij  =  =  a2hjLRLThj, 

then 


gyU)  -  Ei|YU)|2} 

»  E{YU)Y*U)} 


N-i  M-!  2_. 

•  E  E  eivj.-J 

n=0  m=0 


N-l  N-l  ,  0 

■  E  E  + <n  -  '> 

n=0  m=0 


-  1£U  (85) 

where  is  defined  as  in  Equation  (82)  and  K  is  an  NxN  symmetric  output 
sequence  covariance  matrix,  the  elements  of  which  are  defined  in  Equation 
(84).  Now  combining  Equations  (82)  and  (85)  results  in  the  expression  for  the 
LMSF  transfer  function.  Let  the  magnitude  square  discrete  frequency  response 
of  the  filter  be  |  H(  2.)  |  z. 

Then 


|HU) 


Gy{£) 


hRh 


0, 


N  -  1 


where  the  matrix  Q  can  be  written  equivalently  as 

q  =a2K  =  hlrlV 


(86) 
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It  should  be  mentioned  that  the  frequency  response  of  the  LMSF  in  general 
depends  on  the  input  spectrum  or  equivalently  the  input  correlation  matrix. 


2.3.6  Alternate  Expression  of  the  Exact  Correction  Factor 


It  was  shown  in  Section  2.2,  via  an  intuitive  frequency  domain  argument, 
that  the  unbiased  correction  factor  of  the  variance  estimate  should  be  related 
to  the  ratio  of  the  output  to  input  bandwidths  (Equation  (20)).  In  Sec¬ 
tion  2.3.4,  the  exact  correction  factor  for  a  finite  input  sequence  (Equation 
(73))  was  derived.  In  Section  2.3.5,  the  frequency  response  of  the  LMSF  was 
calculated.  It  is  intended  in  this  section  to  show  that  the  exact  correction 
factor,  as  derived  in  Section  2.3.4,  can  be  related  to  the  ratio  of  output  to 
input  bandwidths  and  therefore  validate  the  intuitive  results  as  presented  in 
Section  2.2  . 


Comparison  between  Equations  (20)  and  (73)  show  that  the  following 
equivalency  must  be  established: 


where  BQ  and  Bi  are  the  equivalent  bandwidths  of  the  output  and  input 

sequences,  respectively.  Let  {X^ } ,  1  =  0 . N  -  1  be  the  input  sequence, 

{Y i } ,  i  =0,  ....  N  -  1  be  the  LMSF  output  sequence.  Furthermore,  let  {X(i)} 
and  { Y ( a. ) }  be  the  OFT  of  the  input  and  output  sequences,  respectively.  Then 
the  equivalent  input  and  output  bandwidth  can  be  written  as: 


N-l 

£  E[|XU)|*] 
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Substituting  Equations  (90)  to  (93)  into  Equations  (88)  and  (89)  yields: 

B .  =  N  T-£aclffl  Af  (94) 

i  "  sum[R]  y  1 

B  =  N  -iLacelHLRI  Af  (95) 

sumfHLRL  H  ] 

and  their  ratio 

B0  =  Trace [H(HTH)'1HTRl  sum [R] 

^7  Trace  [R]  sum(HLRLTHT] 

=  Trace  fH(HTH)~1HTR1  sum  [R1  (96j 

N  sum [HLRLTHT] 

since  R  is  a  normalized  correlation  matrix  Trace[R)  =  N.  Comparing  Equation 
(96)  to  Equation  (87)  shows  that  the  desired  result  exists  if,  and  only  if, 
the  following  equality  holds: 

sum  [R]  =  sum[HLRLTHT]  (97) 

It  can  be  shown  that  equality  in  Equation  (97)  does  hold  for  all  values  of  R 

(Appendix  B).  Therefore,  the  equivalency  in  Equation  (87)  is  established. 


2.4  ERROR  ANALYSIS  OF  THE  UNKNOWN  MEAN  MODEL 

Up  to  this  point,  only  the  correction  factor  for  the  LMSF  acting  on  the 
noise  process,  n(t),  has  been  analyzed  and  derived.  In  practice,  the  choice 
of  fitting  function  chosen  for  the  LMSF  will  not  match  the  unknown  mean,  m(t), 
exactly.  Thus,  the  residuals  of  the  mismatch  between  the  unknown  mean  m(t) 
and  the  fitting  functions  will  contaminate  the  variance  estimate  of  the  noise 
process,  n(t). 
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Now  to  be  examined  is  the  error  caused  by  the  correction  factor,  C  (Equa¬ 
tion  (24),  which  was  derived  for  noise  input  only),  on  the  variance  estimate 
of  the  noise  process  n(t)  when  the  actual  input  consists  of  both  the  unknown 
mean,  m(t),  and  the  noise  process,  n(t).  In  addition,  the  effect  on  the  error 
by  the  choice  of  fitting  functions  will  be  commented  upon. 

Assume  that  the  expected  variance  of  the  residual  error  of  an  LMSF  due  to 
the  mismatch  in  the  assumed  modeling  of  the  unknown  mean  process  m(t)  is  given 
by  o|.  Also  assume  the  expected  variance,  a*,  of  the  residual  error  of  an 
LMSF  due  to  the  noise  process,  n(t),  is  related  to  the  actual  variance,  o^,  of 
the  noise  process  n(t)  by  the  correction  factor,  C.  Then  the  error  in  esti¬ 
mating  a by  multiplying  the  expected  variance  of  the  residuals  of  an  LMSF  due 
the  process  m(t)  +  n(t)  by  the  correction  factor,  C,  is  given  below. 

Error ^  =  C  •  [c|  +  o*]  -  a*  (98) 

or 

Error ^  =  Ca| 

The  correction  factor,  C,  corrects  the  zero-mean  noise  component  of  the 
expected  variance  of  the  LMSF  residual  due  to  the  actual  input  zero-mean  noise 
variance.  However,  the  correction  factor,  C,  will  amplify  the  error  component 
of  the  expected  variance  of  the  LMSF  residuals  due  to  the  mismatch  in  the 
modeling  of  the  unknown  input  mean.  Thus,  the  above  error  results. 

If  no  correction  was  performed,  the  error  in  estimating  the  variance  of 
the  input  zero-mean  noise  process  n(t)  from  the  expected  variance  of  the  LMSF 
residuals  is  given  by: 


Error,,  =  ^  .  CT2  {99) 

or 

Error j  +  a*  •  fl  -  C] 
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Let  o|  be  modeled  as  a  factor,  a,  times  a* 

o|  *  a  •  a*  (100) 

A  solution  can  now  be  conducted  for  a  to  determine  what  percentage  cr|/a*  must 
be  for  the  correction  factor,  C,  to  yield  a  smaller  absolute  error.  This  can 
be  done  by  finding  the  break  even  point  between  the  two  error  expressions. 

Squaring  and  equating  Equation  (98)  and  Equation  (99)  and  using  Equation  (100) 
yields  the  fol lowing: 


(a C)2(^)2  -  (1  -  C  +  ct)2(^)2 


(101) 


or 


(C2  -  l)a2  +  2(C  -  l)a  -  (C  -  l)2  =  0 


The  positive  solution  of  Equation  (101)  is: 


a 


-  C  -  1 

-  Tm 


Using  the  correction  factor  of  Equation  (24)  yields: 


K  +  1  n 

/ 

K  +  1 

N  •  At  •  8Wg( input) 

/ 

N  •  At  •  BWg( input) 

or>  assumin9  --sr."5ie()npUt)  «i, 

_  _ K  +  1 _ 

3  ~  2  •  N  «  At  •  BWg( input) 


(102) 


(103) 


For  the  purpose  of  illustration,  assume  nominal  values  of  K  =  6,  At  =  1, 
8We( input)  =  1  and  N  =  250.  It  yields  the  following  a: 


a  =  .014 
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From  the  above,  it  is  obvious  either  that  the  ratio  of  the  input  variance 
of  the  unknown  time  varying  mean  to  the  input  variance  of  the  zero-mean  noise 
must  be  very  small  or  that  the  LMSF  filter  will  proportionately  remove  more  of 
the  unknown  mean  than  the  noise  or  both.  Fortunately,  the  near-stationary 
conditions  under  which  the  random  bearing  error  tests  are  conducted  tend  to 
insure  both  of  the  above. 

As  noted  earlier  for  an  input  white  sequence,  and  a  fact  which  is  still 
valid  for  reasonable,  correlated  input  sequences,  the  effect  of  the  LMSF  on 
the  noise  process  n(t)  is  independent  of  the  form  of  the  fitting  functions. 
However,  the  form  of  the  fitting  function  can  have  a  pronounced  effect  on  the 
removal  of  the  unknown  mean  process  m(t).  For  example,  if  the  unknown  mean 
was  a  sinusoid,  and  the  fitting  function  contained  that  sinusoid,  the  expected 
variance  of  the  LMSF  residuals  due  to  the  mean  would  be  identically  zero. 
However,  if  the  fitting  function  did  not  contain  the  exact  sinusoid  there 
would  be  an  error,  and,  if  the  fitting  function  was  a  polynomial,  the  error 
would  be  still  larger. 

If  the  correction  factor,  C,  is  applied  to  the  residuals  of  the  LMSF, 
then  Equation  (98)  conveys  the  information  that  the  expected  error  is  always 
positive  and  nominally  depends  only  on  the  expected  variance  o|  of  the  LMSF 
residuals  due  to  the  unknown  mean  process  m(t).  Therefore,  if  several  forms 
of  the  fitting  function  are  used  to  estimate  the  variance  of  the  noise  pro¬ 
cess,  n(t),  after  applying  the  correction  factor,  C,  the  smallest  estimate  is 
the  best. 
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3.0  COMPUTER -GENERATED  DATA 


A  VAX  11/780  computer  was  used  to  compute  the  theoretical  power  spectrum 
and  correction  factor  for  the  LMSF  procedure.  The  computations  were  performed 
using  the  theoretical  equations  of  Section  2.0,  with  several  selected  input 
parameters.  In  addition,  computer  simulations  were  conducted  to  verify  the 
theoretical  predictions  and  to  compare  the  old  LMSF  random  error  procedure 
with  the  new  procedure.  The  following  paragraphs  describe  and  illustrate  the 
computer-generated  data. 


3.1  THEORETICAL  DATA 

Assuming  a  white  input  sequence.  Equation  (59)  can  be  used  to  calculate 
the  theoretical  discrete  power  spectrum  for  a  polynomial  LMSF  filter  (i.e., 

H.j  =  t?"*  i  -  1  to  N;  j  *  1  to  K  +  1).  Figure  2  illustrates  the  resulting 
discrete  power  spectra,  assuming  100  samples  (N)  sampled  at  1  second  (At)  and 
K  =  2,  4,  6  the  order  of  the  polynomial  LMSF.  As  can  be  seen  from  Figure  2, 
increasing  the  order  of  the  polynomial  fit  proportionately  increases  the  pass 
band  of  the  polynomial  LMSF  filter.  In  all  cases,  the  rolloff  is  approxi¬ 
mately  6  dB/octave  at  the  low-frequency  region  and  the  equivalent  two-sided 
bandwidths  are  exactly  given  by  (K  +  l)/(NAt). 

Figures  3  and  4  show  the  theoretical  discrete  power  spectrum  for  a  sin/ 
cos  LMSF  filter  (i.e.,  =  1  for  j  =  1;  =  sin(^  j  •  t^)  for  j  =  2,  4,  6, 

...;  H.^.  =  cos(jj  (j  -  1 )  t  ^ )  for  j  =  3,  5,  7,...;  i  =  1  to  N;  J  =  1  to  K  +  1 
where  K  must  be  even).  Similar  to  Figure  2,  the  number  of  samples  (N)  was 
held  constant  at  100,  the  sampling  time  (At)  was  1  second  and  the  order  of  the 
fit  (K)  was  set  to  4  and  6,  respectively.  From  Figures  3  and  4,  the  equiva¬ 
lent  two-sided  bandwidth  is  obviously  given  by  (K  +  l)/(NAt),  which  is  the  same 
as  the  polynomial  LMSF  filter.  However,  the  rolloff  is  infinite  as  com¬ 
pared  to  the  -6  dB/octave  for  the  polynomial  LMSF  filter.  From  the  foregoing, 
it  is  obvious  that  although  all  LMSF  filters  have  identical  equivalent  band- 
widths  (input  white  sequence),  the  fine  structure  of  their  response  is  highly 
dependent  on  the  choice  of  the  fitting  functions. 
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Figure  5  shows  the  theoretical  discrete  power  spectrum  (white  input 
sequence)  for  a  polynomial  LMSF  filter  where  K,  the  order  of  the  polynomial 
fit,  is  fixed  at  6  and  N,  the  number  of  samples,  were  set  at  50,  75  and  100. 
The  sampling  time  was  1  second.  From  Figure  5  it  can  be  seen  that  the  equiva¬ 
lent  bandwidth  of  the  discrete  power  spectrum  is  inversely  proportional  to  the 
number  of  samples,  N. 

A  parameter  of  practical  interest  is  the  ratio  (r)  of  the  equivalent 
bandwidth  of  the  LMSF  filter  to  the  input  equivalent  bandwidth  (BWg)  defined 
below. 


r  * 


K  +  1 

N  •  At  •  BW 

e 


(104) 


The  input  bandwidth  8Wg  can  often  be  accurately  estimated.  Therefore,  r  can 
be  used  to  relate  the  parameters  of  the  LMSF  filter  (K,  N)  to  the  input  band¬ 
width  (BWg).  in  Equation  (24),  the  approximate  correction  factor  between  the 
sum  of  squared  residuals  of  the  LMSF  and  input  variance,  can  be  expressed  in 
terms  of  r. 


Approximate  Correction  Factor  =  ^  _  r 

Figures  6,  7  and  8  illustrate  the  theoretical  percentage  error  in  the 
standard  correction  factor  (Equation  (10))  for  white  input  sequence  and  the 
approximate  correction  factor  (Equation  (24))  for  correlated  input  sequence 
versus  r  for  K,  the  order  of  the  polynomial  LMSF  equal  to  0,  3  and  6,  respec¬ 
tively.  The  number  of  samples  (N)  was  held  constant  at  100  and  the  sampling 
time  was  1  second.  The  correlated  noise  is  modeled  as  the  sampled  output  of  a 
simple  one-pole,  low-pass  filter  to  a  unit  variance  white  noise  input.  The 
correlation  function  for  the  correlated  input  sequence  is  given  by: 

where  a  is  the  filter  coefficient  (106) 
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and  the  two-sided  equivalent  bandwidth  is  given  by  (see  Appendix  C): 

BWe  =  (1  -  o)/(l  +  a)  (107) 

Equation  (73)  is  used  to  calculate  the  exact  correction  factor. 

Figure  6  confirms  the  earlier  stated  fact  that  the  approximate  correc¬ 
tion  factor  is  exact  for  K  =  0.  Although  Figures  7  and  8  show  that  the 
approximate  correction  factor  for  r  >  .5  has  a  greater  percentage  error  than 
the  standard  correction  factor,  the  approximate  correction  factor  is  signifi¬ 
cantly  better  than  the  standard  correction  factor  in  the  useful  region  where 
r  .25.  A  good  rule  of  thumb  would  be  to  choose  the  parameter  (N,  At,  K)  of 
the  IMSF  such  that  r  remained  less  than  .25  for  a  given  correlated  input  noise 
sequence. 

Since  the  LMSF  filter  is  dependent  on  the  input  data,  the  power  transfer 
function  will  also  be  a  function  of  the  input  data.  Therefore,  it  is  obvious 
that  the  power  transfer  function  of  the  LMSF  filter  will  also  be  dependent  on 
the  correlation  or  equivalent  bandwidth  of  the  input  sequence.  However,  if 
the  equivalent  bandwidth  of  the  LMSF  filter  is  small  compared  to  the  equiva¬ 
lent  bandwidth  of  the  input  sequence,  the  pass  band  and  initial  rolloff  of  the 
LMSF  filter  power  transfer  function  will  be  only  slightly  affected.  Figures  9 
and  10  show  the  comparison  between  the  power  transfer  function  for  an  input 
white  sequence  and  an  input  correlated  noise  sequence  (r  =  .25).  Figure  9  is 
for  a  3rd  order  polynomial  fit  and  Figure  10  is  for  a  6th  order  polynomial 
fit.  In  both  cases,  the  number  of  samples  was  100  and  the  sampling  time  was 
1  second.  The  correlated  input  sequence  was  the  output  of  a  simple,  low-pass 
filter  to  a  white  noise  input.  From  examining  Figures  9  and  10  it  is  obvious 
that  the  power  transfer  functions  for  the  input  white  and  correlated  input 
sequences  are  identical  until  -15  dB,  where  the  correlated  power  transfer 
function  flattens  out.  As  expected,  there  is  no  change  in  the  power  transfer 
function  of  the  zero  order  polynomial  LMSF  filter  with  correlated  input  data. 
More  surprising  is  that  the  power  transfer  function  for  the  sin/cos  LMSF 
filter  is  not  a  function  of  the  correlation  of  the  input  sequence  for  any 
order  fit.  In  summary,  the  power  transfer  function  is  dependent  on  the  cor¬ 
relation  of  the  input  noise  sequence  and  the  fitting  function  of  the  LMSF. 
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3.2  SIMULATION  OATA 

In  order  to  verify  the  theoretical  prediction,  Monte-Carlo  simulations 
were  performed  on  selected  theoretical  predictions. 

Figures  11  and  12  compare  the  simulated  and  theoretical  power  transfer 
function  with  a  white  noise  input  for  a  3rd  and  6th  order  polynomial  LMSF 
filter,  respectively.  The  simulated  power  transfer  function  was  calculated 
by  taking  the  FFTs  of  the  input  psuedo-random  Gaussian  sequence,  and  the  cor¬ 
responding  output  sequence.  The  output  power-transfer  function  consists  of 
taking  10  log  of  the  ratio  of  the  output  to  the  input  magnitude  squared  of  the 
FFT  data.  The  simulation  was  repeated  and  averaged  1000  times.  Figures  11 
and  12  indicate  a  reasonable  agreement  between  vhe  simulation  and  the 
theoretical  results. 

Figure  13  compares  the  simulated  and  theoretical  percentage  error  in  the 
approximate  correction  factor  (Equation  (105))  as  a  function  of  r,  the  ratio  of 
the  equivalent  bandwidth  of  the  LMSF  filter  and  the  equivalent  bandwidth  of 
the  correlated  input  sequence.  The  simulation  is  for  a  6th  order  polynomial 
fit  consisting  of  128  samples  sampled  at  a  1-second  rate.  The  correlated 
input  sequence  was  formed  by  passing  psuedo-Gaussian  random  number  (Xi ) 
through  a  simple  digital  filter,  given  below: 

Yi  -  aY^j  +  (1  -  a)X.  (108) 

where  a  is  a  filter  parameter. 

In  the  limit  (infinite  samples),  the  equivalent  input  bandwidth  is  given 
by: 


BWinput  =  ^  •  M1  +  «>  U»> 

The  expected  value  of  the  simulated  correction  factor  was  calculated  by  aver¬ 
aging  the  squared  residuals  of  a  6th  order  polynomial  LMSF  1200  times.  The 
actual  correction  factor  was  determined  by  measuring  the  input  and  output 
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variances  for  153,600  samples  and  then  dividing  the  input  variance  by  the  out¬ 
put  variance.  As  can  be  seen  from  Figure  13,  the  simulated  data  agrees  quite 
well  with  the  theoretical  predictions. 
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4.0  NEW  RANDOM  ERROR  CALCULATION  TECHNIQUE 


The  following  outline  summarizes  the  new  technique  in  tracker  random 
error  calculation. 

1.  Perform,  in  parallel,  both  a  polynomial  and  sin/cos  LMSF  on  the  bear¬ 
ing  error  data  in  order  to  remove  the  unknown  mean. 

2.  In  order  to  obtain  a  good  statistical  average,  choose  N,  the  number 
of  samples,  to  be  as  large  as  possible  under  the  constraint  that  the  indicated 
SNR  remains  relatively  constant  over  the  entire  interval.  The  above  should 
ensure  that  the  tracker  bearing  data  is  a  valid  function  of  the  averaged  SNR. 
In  addition,  to  ensure  numerical  and  statistical  validity,  constrain  N  to  fall 
between  60  <  N  <  600.  At,  the  sampling  rate,  is  set  by  the  data  rate  to  be 

1  second. 

3.  Maintain  the  ratio,  r,  of  the  equivalent  LMSF  bandwidth  to  the  equiv¬ 
alent  input  bandwidth  at  a  value  less  than  .25  by  selecting  the  order  of  the 
LMSF  using  the  following  equation: 

K  =  Nearest  Integer [r  •  BWg  •  N  •  At  -  1]  (110) 

If  K  <  0,  then  K  =  0 
If  K  >  6,  then  K  =  6 

where  r  =  .2. 

BWg  =  is  the  estimated  two-sided  bandwidth  of  the  input  data 
N  =  the  number  of  samples 
At  =  1 

K  =  the  order  of  the  LMSF. 


51 


TR  6601 


Por  the  sin 'cos  IMSF,  K  is  rounded  to  the  nearest  even  integer  (0,  2,  4,  6). 

4.  In  order  to  remove  the  bias  in  the  variance  measurement  resulting 
from  the  LMSF,  multiply  the  squared  residuals  of  the  LMSF  by  the  following 
correction  factor: 

correction  factor  =  - ■« - —  (111) 

(«  •  At  •  8Ue) 

the  equivalent  two-sided  input  bandwidth  can  be  estimated  from  the  averaged 
SNR. 


5.  Select  the  smaller  of  the  two  variance  estimates  resulting  from  the 
polynomial  and  sin/cos  LMSFs. 

6.  Associate  the  resulting  bearing  variance  estimate  with  the  linear 
averaged  SNR  over  the  interval. 

As  a  final  test,  a  simulation  was  performed  comparing  the  new  random 
error  technique  to  the  previous  random  error  technique  (para.  1.1),  using  a 
model  of  a  modern  sonar  broadband  tracker  to  generate  bearing  data.  The  simula¬ 
tion  was  performed  assuming  that  the  unknown  mean  was  zero.  Therefore,  the 
simulation  tested  only  the  ability  to  correctly  estimate  the  input  variance 
from  the  sum  of  squared  residuals  of  the  LMSF.  However,  in  general,  the 
higher  the  order  of  the  LMSF  the  better  it  can  remove  the  unknown  mean  and 
thus  yield  an  accurate  random  error  measurement.  The  simulation  was  similar 
to  the  one  described  in  paragraph  3.2  except  that  a  model  of  the  actual 
broadband  tracking  loop  was  used  instead  of  the  simple  low-pass  filter  given 
in  Equation  (108).  The  measured  equivalent  bandwidth  and  variance  of  the 
modeled  broadband  tracker  was  in  excellent  agreement  with  the  theoretical 
prediction,  implying  a  good  model.  Figure  14  plots  the  order  of  the  polynomial 
LMSF  selected,  as  well  as  the  percentage  error  for  both  the  new  and  old  random 
error  procedures.  The  simulation  was  performed  using  a  sample  size  of  128  and 
only  polynomial  fitting  functions.  From  Figure  14  it  can  be  seen  that  the 
percentage  error  using  the  new  technique  is  always  better  than  the  percentage 
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error  in  the  old  technique,  and  almost  always  less  than  a  few  percent.  In 
addition,  except  for  a  minor  exception  at  -5  dB,  the  order  of  the  polynomial 
fit  for  the  new  technique  is  always  greater  than  or  equal  to  the  old  technique. 
In  summary,  the  new  random  error  procedure  should  represent  a  significant 
improvement  over  the  old  random  error  technique. 
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5.0  SUMMARY  AND  CONCLUSIONS 


1.  A  Least  Mean  Square  Fit  (LMSF)  procedure  is  used  to  remove  an 
unwanted  and  unknown  time-varying  mean  from  recorded  noisy  and  biased  bear¬ 
ing  error  data.  The  residual  resulting  from  the  LMSF  is  used  to  estimate 
the  bearing  variance. 

2.  The  effect  of  the  LMSF  on  the  time-varying  mean  can  be  analyzed 
separately  from  the  analysis  of  the  effect  of  the  LMSF  on  the  noisy  unbiased 
bearing  error  data;  provided  that  the  zero-mean  noise  data  is  independent  of 
the  time-varying  mean. 

3.  The  LMSF  can  be  modeled  and  analyzed  as  a  low-pass  filter  acting  on  a 
white  or  correlated  noise  input  sequence. 

4.  The  calculation  of  variance  based  on  LMSF  residuals  is  known  to  be 
biased  and  an  appropriate  correction  factor  is  needed.  For  the  case  of 
uncorrelated  uata,  the  correction  factor  is  known  and  is  determined  by  the 
order  of  the  fit,  the  number  of  samples  and  the  sampling  interval.  It  was 
shown  in  this  study  that  the  correction  factor  can  be  equated  to  the 
equivalent  bandwidth  of  the  LMSF  when  treated  and  analyzed  as  a  filter.  This 
study  extends  the  classical  correction  factor  to  include  the  case  of  correlated 
data.  The  exact  correction  factor  has  been  derived  and  is  shown  to  be  related 
to  the  covariance  matrix  of  the  input  sequence  (Equation  (73)). 

5.  An  excellent  approximation,  which  depends  on  the  estimate  of  the 
input  equivalent  two-sided  bandwidth,  has  been  derived  which  corrects  the  sum 
of  squared  residuals  of  the  LMSF  to  an  unbiased  estimate  of  the  desired  input 
variance. 


6.  Based  on  the  theoretical  analysis  of  the  LMSF  filter  and  the  derived 
correction  factor,  a  new  technique  has  been  developed  and  applied  to  estimate 
tracker  random  error.  This  new  procedure  shows  substantial  theoretical 
improvement  over  the  previous  procedure. 
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Figure  1.  Illustration  of  Input  Variance  to  Output  Variance  Relationship 
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FIGURE  2  LMSF  POWER  VS  FREQUENCY 
180  SAMPLES 


FIGURE  3  LMSF  POWER  VS  FREQUENCY 

4th  ORDER  SINXCOS  FIT  100  SRMPLES 
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FIGURE  12  LSMF  POWER  ■> -j  FREQUENCY 

6th  ORDER  POLYNOMIAL  FIT  64  SAMPLES 
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FIGURE  13  H  ERROR  OF  NEW  CORRECTION  FRCTOR  VS  RATIO  OF  BRNDWIDTHS 
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APPENDIX  A 

PROOF  OF  sum[H(HTH)-1HT]  =  N 


Recall  that 


H  = 


1  t 
1  t 


1  ’• 
2 


1  t. 


N  J 


(A-l) 


is  an  NxM  matrix,  where  N  equals  the  sample  size  and  M  =  K  +  1  where  K  is  the 
order  of  the  fit.  For  the  problem  to  be  meaningful  we  must  have  M  «  N.  Also 
recall  that  A  =  H(HTH)"lHT  is  an  NxN  square  symmetric  matrix,  and  the  matrix  is 
related  to  the  least  square  solution  of 

Z  =  Ha  +  n  ( A— 2 ) 

where  Z  is  the  N  dimensional  measurement  vector,  a  is  an  H  dimensional  coef¬ 
ficient  vector,  and  n  is  the  noise  vector.  The  least  square  solution  of 
Equation  (A-2)  yields: 


a  =  (HTH)-1HTZ 

and  the  best  estimate  of  Z  is  given  by 

Z  =  Ha  =  H(HTH)_1HTZ  =  AZ 


(A-3) 


(A-4) 
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N  N 

For  a  given  matrix  A,  the  notation  sumfA]  =  J  A^.;  i.e.,  the  sum  of  all 

1  ^  T  -  T 

the  elements  in  A.  We  want  to  show  that  given  A  =  H(H  H)"lH  ,  sumfA]  *  N, 
where  N  is  the  dimension  of  A. 


Using  basic  matrix  manipulations,  it  is  easy  to  establish  the 
equivalence 


sumfA]  =  1TA1  (A-5) 

where  lj  =  [1  1  1  . . .  1] ,  a  row  vector  of  N  ones. 

N 

Denote  the  N  dimensional  real  space  by  R  ,  the  M  dimensional  real  space 
by  RM,  then  for  all  possible  values  of  Z,  the  measurement  vector,  and  £,  the 

M 

noise  vector,  the  difference  vector  d  *  Z  -  n  is  contained  in  the  space  D 

N 

which  is  a  subspace  of  R  ;  i.e.,  one  can  write 

0N  ■  {d:d  =  Ha  V  aERM} 

The  space  DN  is  a  subspace  of  RN  because  the  Rank [H]  <  N.  Denote  the  ith 
column  of  H  by  hi . ,  then  one  can  write 

d  =  Ha 

=  [hx  hg  ...  hMJ 


M 

=  2 
i  =  l 

Thus  d  is  a  linear  combination  of  h- .  In  fact,  elements  of  the  set  {M  1*1, 

N  ~ 

2,  . ...  M  are  linearly  independent  and  span  the  space  D  ,  therefore  the  set 
forms  a  basis.  However,  base  vectors  are  not  unique.  In  particular,  there 


(A— 6) 


A- 2 
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exists  an  orthonormal  basis;  i.e.,  the  base  vectors  are  orthogonal  to  each 
other  and  have  unity  length.  Given  the  basis  ,  an  orthonormal  set  can  be 
constructed  using  the  Grahm-Schmidt  orthogonal ization  procedure. 


denotes  the  inner  product  norm  and  defines  the  following  notations: 

*  il  = 


h  =  -i 


Hi 


k  =  h2  -  ^—2  *  Sl>*l 


*  -2  = 


h 

k 


M 


(A— 7) 


M-l 


Im  =  hM  '  S  %  *  li)li  ;  =  liT 


Im 


i=l 


m 


then  the  set  {e^. }  =  j  -jJ 1  ^  •  |  i  «  1,  2,  . ..,  M  is  the  required  orthonormal 
set.  The  relation  betweer’i  {e i }  and  {h^ }  can  be  established  using  Equation 
(A-6)  since  each  e^  is  a  linear  combination  of  the  set  {h^}.  Thus,  we  have  the 
relation 


t®-l  ®2  ***  =  —2  ••• 


P11  P21  "•  PM1 


0 

0 


P22  PM2 


0  P 


2M 


MM 


or  in  matrix  form 


E  =  HP. 


(A-8) 
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Note  pn  =  ikf  ’  where  “-I11 

Since  {e . }  is  an  orthonormal  set, 


3  N  denotes  the  inner  product  norm  of  h^. 
we  have  the  obvious  relation 


ETE 


(A-9) 


Note  that,  since  P  is  a  basis  transformation  matrix,  it  is  non-singular,  and 
its  inverse  exists  [4].  Post  multiplying  Equation  (A-7)  by  P"1  yields 

H  3  EP-i  (A- 10) 

Substituting  Equation  (A-10)  into  Equation  (A-3)  yields 

a  3  (HTH)'lHTZ 

3  [(P'1)TETEP'1]'1(P‘1)TETZ 

3  PETZ  (A-ll) 

T  —1 

Now  since  h^  *  [1  1  1  ...  1]  «  _l,  and  e^  3  ^  ,  we  have  1  3  Ne^.  Let  Z  3  1_  in 

Equation  (A-ll),  we  obtain 


P11  P21  PM1 

*1*1 

o  p22  ...  pM2 

*2*1 

•  •  • 

• 

•  •  « 

• 

•  •  • 

• 

0  P2M  ***  PMM 

*M*1 
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I  p 

N  yZl 
0  P„ 


( A- 12) 


Now  using  Equation  (A-12)  in  Equation  (A-4),  we  have  the  desired  result 


Ha  =  1 


(A— 13) 


sum[A]  =  1  Al^ 


=  1TH(HTH)-1HT1 


T  ~ 

=  1  Ha 


Q.E.D. 
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APPENDIX  B 

PROOF  OF  sum[H(HTH)-1HTRH(HTH)-1HT]  =  sum[R] 


Recall  from  Appendix  A  that  the  matrix  H  is  given  by  Equation  (A-l)  and 
that  R  is  the  covariance  matrix  of  the  input  sequence.  Also  recall  that  the 
notation  sum  [A]  denotes  the  element  sum  of  the  matrix  A. 

Now 


sum[R]  =  _1^R_1 

where  1^  *  [1  1  ...  1]  a  row  vector  of  ones. 

Then  using  results  of  Equations  (A-l)  and  (A-13)  from  Appendix  A,  we 
obtain 

sum  [H(HTN)'1HTRH(HTH)'lHT] 

=  irH(HTH)"lHTRH(HTH)"lHTl 
=  [H(HTH)"1HTl]TR[H(HTH)_lHTl] 

■  1TR1 
=  sum[R] 


since 

H(HTH)'lHTl  =  1 
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APPENDIX  C 

EQUIVALENT  BANDWIDTH  OF  A  FIRST  ORDER  DIGITAL  FILTER 


Given  a  first  order  digital  filter 


Yi  =  o iful  +  (1  -  a)Xi 

with  i  =0,  1,  2,  . . . ,  0  <  a  <_  1  and  =  0  for  i  <  0. 


Taking  the  Z-transform  of  Equation  (C-l)  yields 


Y(Z)  =  aZ*xY(Z)  +  (1  -  a)X(Z) 


and  the  transfer  function  is 


H(Z)  = 

X(Z)  1  -  otZ'1 

The  equivalent  bandwidth  of  the  transfer  function  is 


BWe  =TTOT 


ITT  / 


H(Z)H*(Z)Z_1dZ 


j  H(Z)H*(Z)Z~1dZ 

urflt 

circle 


since  |H(1) 


|2  = 


I  -  g 
1  -  oZ" 


=  1. 


Z=1 
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(C-l) 


(C-2) 


(C-3) 


( C— 4 ) 


C-l 
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Now  H(Z)  =  - 1-:  — —  which  has  a  pole  at  ZD  =  a  <  1  and  H*(Z)  =  —  which 
1  -  aZ" 1  1  -  aZ 

has  a  pole  at  Zp  =  ^  _>  1,  thus  H( Z)  has  a  pole  inside  the  unit  circle  while 
H*(Z)  has  a  pole  outside  the  unit  circle.  Therefore  using  the  residue 
theorem,  we  have 


H(Z)H*(Z)Z"1dZ 


*  t(Z  -  Zp)H(Z)H*(Z)Z'l]z=Zp 


Z=a 


.  1  -  a 


(C-5) 


Therefore  the  equivalent  bandwidth  is 

BWe  *  T"r?  *  IF  (Hertz)  (C-6) 

where  At  is  the  sampling  interval. 
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